Learning Real-Time Stereo Vergence Control

On-line learning robotic systems have many desirable properties. This work contributes a reinforcement learning framework for learning a time-constrained closed-loop control policy. The task is to verge the two cameras of a stereo vision system to foveate on the same world feature, within a limited number of perception-action cycles. On-line learning is beneficial in at least the following ways: (1) The control parameters are optimized with respect to the characteristics of the environment actually encountered during operation; (2) visual feedback contributes to the choice of the best control action at every step in a multi-step control policy; (3) no initial calibration or explicit modeling of system parameters is required; (4) the system can be made to adapt to non-stationary environments. Our vergence system provides a running estimate of the resulting verge quality that can be exploited by a real-time scheduler. It is shown to perform superior to two hand-calibrated vergence policies.